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Abstract 

While MOOCS have emerged as a new form of open online education around the world, research 
is still lagging behind to come up with a sound theoretical basis that can cover the impact of socio¬ 
economic background variables, ICT competences, prior experiences and lifelong learning profile, 
variance in intentions, environmental influences, outcome expectations, learning experience, and 
economic return on taking and completing Massive Open Online Courses (MOOCs). The potential 
diversity of participants of MOOCs has been taken as a starting point to develop a theoretical 
model and survey instrument with the goal to establish a large-scale, cross-provider data 
collection of participants of (European) MOOCs. This article provides an overview of the 
theoretical model, the start-phase of the project, and reflects on first experiences with the cross¬ 
provider data collection. 


1 The views expressed in this article are purely those of the author and may not in any circumstances be 
regarded as stating an official position of the European Commission. 
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Introduction and Related Work 

Open educational formats have received more attention recently due to the hype around Massive 
Open Online Courses (MOOCs). Institutions all around the world are joining initiatives to provide 
external participants cost-free and barrier-free access to their online-courses. Open Universities 
around the world already have a tradition in supporting open learning practices for people who 
are unable to follow traditional formats of educational programs provided by higher educational 
institutions. We have earlier discussed the frictions and gaps in the discussion about MOOCs and 
have reframed it with regard to relations to the open educational resource initiatives, experiences 
with open educational practices and the potential benefit of learning network services for MOOCs 
(Kalz & Specht, 2013). In a meta-review Liyanagunawardena, Adams, and Williams (2013) 
summarize existing research on MOOCs up to 2012. The authors state that most studies to date 
have focused on case studies, the influence of MOOCs in higher education structure, or 
educational theory framing. Although MOOCs generated a plethora of data, the learner 
perspective is still underrepresented in current research. Fischer (2014) argues that, currently, we 
are still in an early development stage of MOOCs and he states that “both the hype and the 
underestimation [of MOOCs] are more based on assumption and beliefs than theoretical 
groundings and qualitative and quantitative data” (p.150). The situation has partially improved 
recently with several studies on MOOC participants using learning analytics or survey 
methodology (MOOCs@Edinburgh Group, 2013; Anderson, Huttenlocher, Kleinber, & Leskovec, 
2014; Christensen, Steinmetz, Alcorn, Bennet, Woods, & Emanuel, 2014) and partially even open 
datasets mainly focused on U.S courses ( Ho, Reich, Nesterko, Seaton, Mullaney, Waldo, & 
Chuang, 2014; Jordan, 2014). 

Recently, more empirical research has been conducted related to the learner perspective in 
MOOCs. This empirical research has treated topics like background of students and behaviour 
related to performance (DeBoer, Stump, Seaton, & Breslow, 2013), intentions of MOOC learners 
(Reich, 2014, Campbell, J., Gibbs, Najafi, & Severinski, 2014), learning paths across several 
MOOCs (Perna, Ruby, Boruch, Wang, Scull, Ahmad, & Evans, 2014) and last but not least 
professional learning in MOOCs (Milligan, & Littlejohn, 2014). These isolated initiatives and 
studies, however, did not provide the joint research instrument that is needed to collect MOOC 
participants’ data across Europe systematically. This large-scale data collection would inform 
both policy-making on a European level, and also higher education provider’s strategic decisions 
with respect to open online education. 

In this article we introduce the theoretical model and survey instrument of the MOOCKnowledge 
project. This project is an initiative of the European Commission’s Institute of Prospective 
Technological Studies (IPTS) with the intention to build a database that can provide insights into 
the profile, experiences and behavior of participants of (European) open online courses. As a 
European project, the diversity of participants and participant profiles is an important focus 
aspect for the project that can only be addressed by a cross-provider data-collection. This 
approach will also enable the researchers to analyse the impact of open education for specific sub- 
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populations, being it minorities, specific professions (e.g. teachers) or learners with specific 
cultural or socio-economic backgrounds. We first introduce in this contribution the European 
policy background of the project. Then we introduce the theoretical background and the research 
focus of the project. We briefly introduce the survey instrument and provide a summary of 
experiences with the cross-provider data-collection. Last but not least, we provide an outlook into 
future research and additional services for MOOC-providers and researchers to be developed in 
the future. 


European Policy and Cultural Background 

The European Commission's overall Europe 2020 strategy highlights the importance of smart, 
sustainable, and inclusive growth to remain competitive and to overcome the current economic 
crisis (European Commission, 2010). The modernization of the European Education and Training 
system, with a special focus on early school leavers and increasing tertiary education attainment, 
is one of the most important means to reach this goal. Linked to the broad objectives of this 
strategy the European Commission (2012) argues in the "Rethinking Education" Communication 
that efforts must be made to boost the full uptake of ICT, enhancing both the acquisition of digital 
competences and the modernisation of education to generate growth, employment, and social 
inclusion. Moreover, this communication emphasized the need to stimulate open and flexible 
learning in order to provide the skills needed in the 21st centuiy society. 

In 2013, the idea of the use of ICT and open and flexible learning in the European educational 
systems was further developed in the Communication “Opening up Education: Innovative 
teaching and learning for all through new technologies and Open Educational Resources” 
(European Commission, 2013). This Communication sets out an “agenda for stimulating high 
quality and innovative ways of learning and teaching through new technologies and digital 
content” (European Commission, 2013). The Communication responded specifically to the recent 
developments in the area of Open Education, such as the abundance of OER and MOOCs, and 
highlighted the need to develop economies of scale and remove barriers to access, use, and 
sharing of knowledge across borders for education. Although it focuses on Open Education, the 
Communication took a systemic approach and made recommendations ranging from general 
topics such as ICT access and skills to specific Open Education topics such as standards, or the 
use and integration of OER and MOOCs in formal and non-formal education. 

In a direct or an indirect way, MOOCs have been signaled by policymakers as a potential solution 
to some of the challenges detected in European educational systems. The reasons are diverse. 
First, MOOCs could potentially expand the access to tertiary education offering free and online 
access to anyone with internet access, including to those who were early school leavers. Second, 
MOOCs could boost the modernisation of education by promoting innovative and efficient uses of 
online education, and by challenging the traditional roles of teachers. Third, MOOCs could be 
used as a means to overcome borders by the educational offer of the universities and to foster the 
interinstitutional collaborations in its development, use, and recognition (EADTU, 2014). The 
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European Credit Transfer and Accumulation system (ECTS) (European Union, 2015) is a valuable 
and distinctive tool which positions Europe in a privileged situation at this regard, enabling the 
transfer and recognition of credits from one organization to the other across borders. Moreover, a 
cross-border MOOC offer guarantees the provision of education following one of the key 
European values: diversity (understood as diversity of culture, languages, methods, levels, topics, 
and profile of learners). Finally, MOOCs could have a positive impact on the skills of the 
individuals and could be useful to foster education and lifelong learning in areas where a lack of 
skills is detected ultimately contributing the generation of growth, employment, and social 
inclusion in the European societies. 

In a recent study Tempelaar, Rienties, Giesbers and van der Loeff (2013) have analyzed the 
impact of cultural differences on learning dispositions. The authors argue that in Europe five of 
ten cultural world clusters are located and that all universities in Europe with an open inflow of 
students face these cultural differences. In their study, the authors show how the learning 
dispositions and learning approaches are influenced by cultural backgrounds of students. 

On the other hand, MOOCs are not free of criticisms. It is often argued that MOOCs are not 
reaching disadvantaged individuals, and therefore these courses could even be contributing to 
increase educational inequalities. Additionally, the prominence of low quality instructional 
designs and high level drop-out rates have been also signaled as problems (Margaryan, Bianco & 
Littlejohn, 2015). Finally, some critics stress that the participation in the MOOC movement by the 
Higher Education institutions could be only a marketing strategy in the global arena, but without 
clear social returns. 

While the potential benefits and risks of the MOOCs have been identified by literature and policy 
makers, there is scarce empirical evidence that allows to assess if they have actually been realised 
or avoided (Fischer, 2014). Since most studies have been focused in U.S context, the lack of 
evidence is specially an issue in the EU context. Therefore, better local knowledge is needed if 
Europe wants to implement effective evidence-based educational policies. Reich (2015) has 
recently criticized that research to date had little impact on educational practice. According to 
him, research needs to change focus and procedures on three levels: from studies of engagement 
to research about learning, from investigations of individual courses to comparisons across 
contexts, and from a reliance on post-hoc analyses to greater use of multidisciplinary, 
experimental design. To overcome the current trend of in-depth single-course or single-provider 
analysis is especially important in Europe due to the cultural, socio-economic, and motivational 
diversity of participants (Kalz & Specht, 2013) . 

The MOOCKnowledge project is embedded in the three policy initiatives described above and 
aims to set up a large scale data collection on participants in European MOOCs. The data 
gathered by the project will allow to assess some of the above mentioned potentials and shortages 
of the MOOCS better on the basis of cross-national and paneuropean evidences. By doing this, the 
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research will take into account diversity of courses and learner profiles in the EU (including 
minority learners), and will contribute to inform better the EU and Member States policies in 
Education. The research model behind the project is introduced next. 

Research Model 

The MOOCKnowledge project addresses directly the underrepresentation of the learners in 
current MOOC research and aims to establish a large-scale cross-provider data collection on 
European MOOC participants. The study tender published by the European Commission’s 
Institute for Prospective Technological Studies (IPTS) defined the background to the research. 
The study contract then further elaborates this basic definition project, which is expected to 
deliver the following types of data on MOOCs participants: 

• socio-economical profile 

• lifelong-learning profile 

• ICT-profile 

• MOOC profile 

• Motivation and intentions 

• Drop-out phenomenon 

On the basis of these components we have developed a research model for the project. In 
addition, two associated PhD projects extend the focus with respect to assessment and feedback 
practices in MOOCs and language learning. 

For this purpose we have used two existing research frameworks. On the one hand, these 
frameworks can guide the construction of the survey instruments based on items validated 
earlier, and on the other hand, they also allow a systematic analysis of the data at a later stage 
(Kreijns, Vermeulen, Kirschner, van Buuren, & Van Acker, 2013; Kreijns, Vermeulen, Van Acker, 
& van Buuren, 2014). These two frameworks are the reasoned action approach elaborated by 
Fishbein and Ajzen (2010) and self-determination theory set out by Ryan and Deci (2000). These 
frameworks offer a basis for the prediction of human social behavior and consist of background 
factors (e.g., socio-economic status) that affect different variables and directly influence the 
behavioural intention to take and complete a MOOC. 

By taking a MOOC we mean that a person enrolls in a MOOC to get access to all the course 
materials and teachers or support services. However, the completion of the MOOC is dependent 
on individual objectives. We define completion in this research model as the achievement of 
earlier set personal objectives, which may not be the same as completion in the sense of 
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completing all learning activities, tests, and finally receiving a certificate. For instance, a MOOC 
taker (i.e., participant) may only be interested in one particular part of a MOOC about statistics, 
namely the part that explains confirmatory factor analyses. From the point of view of the MOOC 
taker, the MOOC is completed if that part is finished whereas the MOOC giver may see this as a 
drop-out. In this sense we define drop-out in the project as the gap between earlier set personal 
objectives and attainment of these objectives during the course. Grau-Valldosera and Minguillon 
(2014) have earlier discussed the complexity of the multidimensional phenomenon of drop-out 
and they have developed a possible definition of drop-out for the distance/online education 
context. Nonetheless, for the open education context, different approaches are needed. We believe 
that our approach to the phenomenon of “drop-out” in open education leads to more expressive 
and also more valid results. 


distal variables proximal variables intention-behavior gap outcomes variables 



Figure 1 : Research model of the MOOCKnowledge project 

Background variables are usually referred to as distal variables whereas the different variables 
that directly influence intention are referred to as proximal variables. Distal variables in our 
model exist on an individual level, a social level, and a task level. These are for example 
demographic data, the socio-economic status of the participants, their lifelong learning profile, 
previous experiences with open online courses and IT competences. The reasoned action 
approach identifies attitude, perceived norm, and perceived behaviour control (i.e., self-efficacy) 
as proximal variables as they directly predict behavioral intention. Attitude is the person’s 
favourable or unfavourable position as regards taking a MOOC. Attitude is generally formed by 
the outcome beliefs of a person and his/her evaluation of these beliefs. For example, a person may 
belief that taking a MOOC will give her/him more opportunities in the labour market and 
evaluate this as very important for that person. Consequently, the person may have a positive 
attitude towards taking a MOOC. 

Perceived norm is the experienced social pressure to take a MOOC. This perceived norm is 
formed by a person’s important others, who may be colleagues, family members, supervisors, etc. 
For example, when a colleague advises a person to take a MOOC, that person may not be 
motivated to comply with this advice. However, when the boss gives the same advise, the same 
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person may feel obliged to comply. Finally, perceived behaviour control defines whether or not a 
person is able to take a MOOC. Does the person have time to take a MOOC? Does she/he possess 
all the necessary equipment and software to follow a MOOC? All these questions will give 
indications to the perceived behavior control. All distal variables exert their influence on a 
person’s intention to take a MOOC through these three proximal variables. In other words, these 
proximal variables are mediating the influence of the distal variables on intention. Another 
element in the reasoned action approach is the intention behaviour gap. Not all intentions will 
result in actual behavior. There may be many reasons: for example, the MOOC may demand some 
prior-knowledge of the topic concerned which a person may not have. This is what Fishbein and 
Ajzen (2010) call actual behaviour control. Actual behaviour control moderates the relationship 
between intention and behaviour. Actual knowledge and skills are also moderate this intention- 
behavior relationship. Actual knowledge and skills refer to all the knowledge and skills that are 
needed to realise the intention. For example, a participant must know which codec has to be 
downloaded to view a footage that is part of the MOOC. 

The more insight a person has about the actual behaviour control the more precise the perceived 
behavioral control will be. Consequently, the person’s intention to perform the behavior will be 
more accurately formed. This, in turn, means that the predictive power of behavioral intention 
will increase. At the same time, the moderating influence of both perceived and actual behavior 
control in the relationship between behavioral intention and actual behaviour will be positively 
altered (see the dotted arrows in Figure 1). 

It is important to notice that until now, nothing has been said about the MOOC itself such as how 
it is organized, what it prerequisites are, if a certificate is obtained after completion, etc. These are 
aspects that the questionnaire will also address. Consistently with the theoretical model the 
project has two questionnaires planned during the course: a pre-course questionnaire which will 
assess the proximal variables and intention, whereas a post-course questionnaire which will also 
assess actual behavior. The reasoned action approach was earlier applied in many different 
domains like health, economics and the authors have applied this approach earlier in an 
educational context, namely to investigate motivations and intentions of teachers to use open 
educational resources (OER)/digital learning materials (DLMs) (Kreijns, Van Acker, Vermeulen & 
Van Buuren, 2013). 

Self-determination theory (SDT) differentiates between intrinsic and extrinsic motivation and 
assumes that people have three basic needs namely: competence, relatedness, and autonomy. 
Autonomy is the feeling that people have when they are originators of their own actions (in 
harmony with the concept of the integrated self). Competence is the feeling people have when 
they are effective, and that there are sufficient opportunities to demonstrate efficacy. Relatedness 
is the feeling people have when they are connected and valued by others and that they experience 
a sense of belonging. If the social environment does not satisfy these basic psychological needs, 
then negative consequences will follow with respect to activity and development. However, the 
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degree to which each of these needs are promoted and satisfied determine the types of 
motivations and the consequent regulations that are developed. These types may be categorized 
as self-determined versus controlled (i.e., non- self-determined) or as autonomous motivation 
versus controlled motivation. The Perceived Locus of Causality (PLOC) (Ryan & Connel, 1989) 
represent all the SDT motivational types ordered in from the most undesirable form of non-self- 
determined behavior (amotivation) to the most desired form of self-determined behavior 
(intrinsic motivation). Figure 2 depicts these types of motivations. 


behavior NON SELF-DETERMINED 

SELF-DETERMINED 

AMOTIVATION 

CONTROLLED MOTIVATION 

AUTONOMOUS MOTIVATION 

type of 

amotivation 

motivation 

extrinsic motivation 

intrinsic motivation 

type of non- 

regulation regulation 

external introjected identified integrated 

regulations regulation regulation regulation 

intrinsic regulation 

... but I have 

no interest in 

MOOCs 

example 

... because ... because I ... because it ... because I am 

my boss would be is important always in to try 

insists that ashamed not to to me to take out new things 

I should take a MOOC a MOOC such as MOOCs 

take a 

MOOC 

... because taking a 

MOOC is fun to me 


degree of 

low 



high 

internalization 

internalization 



internalization 

locus of 

causality 

impersonal external 

somewhat 

external 

somewhat 

internal 

internal 


integrated 


internal 


Figure 2 : The different types of motivation according to SDT 

In short, the different types of motivations are as follows: Autonomous motivation encompasses 
the preferred types of motivation, namely intrinsic, integrated, and identified motivation, the 
latter two are types of extrinsic motivation as well. Controlled motivation encompasses the two 
other types of extrinsic motivation, namely introjected and external motivation. Autonomous 
motivation is associated with self-determined regulation whereas controlled motivation is 
associated with coercive regulation (thus non-self-determined regulation). Intrinsic motivation is 
concerned with the active engagement with tasks solely because one finds them interesting and 
will give fun. Integrated motivation is the type of extrinsic motivation that is the closest to 
intrinsic motivation; it deals with doing the tasks because the activity is in compliance with the 
self. Identified motivation is the type of motivation in which humans identify the activity as 
important and useful and, thus, are willing to perform the activity. Introjected motivation 
represents a regulation that is driven by the feelings of guilt and shame as well as by factors such 
as self-esteem and ego-involvement. External motivation is the strongest type of extrinsic 
motivation in that the regulation of the activity is a function of expectations regarding reward and 
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punishment. Finally, amotivation, also associated with non-self-determined regulation, is the type 
of motivation in which one performs the activity but intention is lacking. 

From the perspective of the MOOCKnowledge project, it is important to have a clear picture about 
motivation because it makes a considerable difference if a participant is motivated intrinsically or 
externally. At the moment, very little is known about the motivational disposition of MOOC 
participants, how motivation is related to their behaviour, and the impact on their study success 
or career development. 

To be able to analyse the intention-behavior gap more thoroughly we have integrated work by 
Gollwitzer about implementation intentions (Gollwitzer & Oettingen, 2013). Implementation 
intentions are concrete plans for how a specific goal can be achieved. This aspect is important, 
because the huge gap between numbers of participants that sign up for a MOOC and the numbers 
who actually start learning activities can point to a MOOC’s low perceived value or lack of 
implementation intentions among potential participants. These three theoretical foundations are 
combined with other aspects like usability aspect or interaction experiences in MOOCs taken. 

In the long-term perspective, the MOOCKnowledge research model also aims to fill another 
identified research gap in the MOOC literature. Currently there are some studies on the 
economics and social returns of higher education and adult learning, but very little on online and 
open learning (Carnoy, Jarillo Rabling, Castano-Munoz, Duart, & Sancho-Vinuesa, 2013). 
Therefore, despite the increasing importance of the MOOC offer, little is known about the effects 
of these courses on formal study success and career development. This is a long term component 
of the MOOCKnowledge research model and the information will be gathered with a follow-up 
questionnaire that will be sent to the learners approximately one year after they have completed 
the post-questionnaire. Although the majority of MOOC learners have a high socioeconomic 
status and educational backgrounds (Hansen & Reich, 2015), it is hoped that the large scale of the 
MOOCs project will also obtain information on the effects of these courses on minority groups 
(such as unemployed people or learners without a degree). The project will also explore variations 
in the economics and educational returns of these courses depending on topic, the country, the 
certification, etc. 

First Experiences with the Survey Instrument 

The survey-instrument of the MOOCKnowledge project was developed in several iterations. The 
pre-questionnaire consists of five chapters each with a special focus. The chapters of the pre¬ 
questionnaire are: 

• Demographic and socio-economic questions 

• Lifelong Learning/Professional Development 

• ICT profile 

This work is licensed under a Creative Commons Attribution 4.0 International License. 


70 



Setting-up a European Cross-Provider Data Collection on Open Online Courses 
Kalz, Kreijns, Walhout, Castano-Munoz, Espasa and Tovar 


• Motivation and intentions 

• Miscellaneous aspects 

The post-questionnaire also consists of five chapters containing again several blocks of items with 
particular focus. The chapters of the post-questionnaire are: 

1. Demographic and socio-economic questions 

2. Learning experience - intention-behavior gap 

3. Feedback (partially optional) 

4. Assessment (partially optional) 

5. Language Learning (optional for specific MOOCs) 

After the post-questionnaire, participants will be recruited for voluntary participation in a follow¬ 
up-questionnaire sent out to participants one-year after finalization of the MOOC. In all chapters, 
reliability and validity is increased by using existing validated instruments, by adapting validated 
instruments, or by developing new instruments following the guidelines of Fishbein and Ajzen 
(2010). 

In an initial analysis we have analyzed data from three different MOOCs. Table 1 provides an 
overview about the high-level differences between learner profiles participating in these MOOCs. 
MOOC A is related with professional training, MOOC B with entrepreneurship and MOOC C with 
personal development. Although some common trends were detected, the data showed how 
different MOOCs were linked to differences among the profile of the learners (Table 1). 

Table 1 


Summary of MOOC Takers Characteristics (11=1244) 



MOOC A 

MOOC B 

MOOC C 

N 

173 

316 

715 

Sex 

Male 56 

Female 117 

Male 194 

Female 122 

Male 248 

Female 467 

Mean age 
(SD) 

43-18 (9.09) 

39-36 (12.28) 

38.04 (12.49) 
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Employment 

employed 

141 

employed 

105 

employed 

289 

status 

self-employed 

15 

self-employed 

80 

self-employed 

57 


looking for work 4 

looking for work 

65 

looking for work 

157 


homemaker 

1 

not looking for work 4 

not looking for work 12 


student 

5 

homemaker 

2 

homemaker 

7 


unable to work 

2 

student 

46 

student 

143 


other 

5 

retired 

2 

retired 

15 




other 

12 

unable to work 

9 






other 

26 

Employer 

no 

16 

no 

20 

no 

42 

support 

yes 

69 

yes 

77 

yes 

100 


does not know 

80 

does not know 

111 

does not know 

271 


NA 

8 

NA 

108 

NA 

302 


Additionally, the data showed some unexpected particularities such as older learners than in US 
studies, high level of unemployed people participation in MOOCs (especially in MOOC B and C), 
or a high percentage of learners who have taken one or more MOOCs previously, pointing to the 
direction that the learners pick and choose from the MOOC offering in order to build personal 
training paths. The relative low importance of certification for appreciation of employees’ 
activities in open online education pointed to a possible increased awareness and appreciation of 
non-formal learning for professional development by employers. 

The current data set at hand had an overrepresentation of participants from Spain. One of the 
methodological challenges of the project will be to come up with an adequate sampling strategy to 
overcome these biases. At the moment, more providers are joining the project and allowing the 
data-collection with the survey instrument of the project. This will contribute to a more 
representative sample. Potential providers can join the project via a cooperation agreement. The 
consortium collects the data for the providers and shares the data with the providers. In the 
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future, the whole data set will be published in an anonymized and aggregated form for other 
researchers to conduct analyses with it. 

The first three MOOCs have also been used to improve the data-collection process and to test an 
adequate length of the questionnaire. With initially around 40 minutes on average and 25 
minutes for the post-questionnaire and a response rate between 5% and 10% the authors will 
explore several options to shorten the survey instrument without losing explanatory power, 
validity, and reliability. 


Discussion and Conclusions 

In this paper we have introduced the theoretical model of the MOOCKnowledge project. We see 
the approach chosen as a promising option to address the shortcomings of the current research 
about MOOCs. The project is open for any MOOC provider or provider of open education with a 
specific focus on the situation in Europe. 

The next step for the project will be to broaden the diversity of providers and cultural contexts 
with the help of the members of the OpenupEd initiative (Roswell & Jansen, 2014) and other 
partners. The more diversity we can achieve on the provider side, the richer our insights will be 
about the MOOC phenomenon on a European and global level. This will also allow the project to 
inform the policy building by the European Commission. Future analysis will focus on participant 
profiles and the relation to achievement, the intention-behaviour gap, or relation between 
employer support and motivation of MOOC participants. 

To provide also feedback to the MOOC providers, at later stages of the project, benchmarking 
options will be explored to compare results of a single MOOC against several other MOOCs and 
data categories. This will enable also a comparison of MOOC investment against initiatives of 
other institutions. In addition, different approaches will be tested how the anonymized and 
aggregated dataset can be explored by the general public, ideally in the form of a set of linked data 
(Piedra, Chicaiza, Lopez, & Tovar Caro, 2014). For this purpose the consortium will also explore 
different data visualization services to be used. 

For these services to become effective, a sufficient amount of data needs to be collected to deliver 
a meaningful benchmarking analysis. Attracting a sufficient amount of MOOC providers will be 
one of the upcoming challenges for the consortium. Potential threats arising from the chosen 
approach are related to a potential selection bias and survival bias. While the selection bias could 
exist on the level of the representativeness of the participating MOOCs, the survival bias will be 
related to the ratio of non-successful participants that fill out both questionnaires. To be able to 
explain the huge gap between subscribed learners and active learners, it will be critical to also 
collect a representative amount of answers from learners who did not reach their learning goal. 
While these problems might be addressed simply by the scale of the data-collection, the 
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consortium has foreseen statistical corrections but also incentive mechanisms to attract higher 
response rates from this special target group. 
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